caffe源码编译 |
您所在的位置：网站首页 › caffe 源码 › caffe源码编译 |
caffe源码编译
|
软硬件环境
ubuntu 18.04 64bit
NVidia GTX 1070Ti
anaconda with python 3.7
CUDA 10.1
cuDNN 7.6
opencv 3.4.2
caffe 1.0.0
简介
先说一下环境，使用anaconda的python虚拟环境，支持opencv，支持CUDA和cuDNN加速，支持在python中调用caffe。基础组件部分可以参考前面的文章，本文就不赘述了 anaconda基本使用 ubuntu安装CUDA和cuDNN opencv源码编译，支持CUDA 基础环境准备安装依赖包和工具 sudo apt install build-essential cmake git ffmpeg libatlas-base-dev libtiff-dev pkg-config python3-dev libavcodec-dev libavformat-dev libswscale-dev libtbb-dev libjpeg-dev libpng-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libx264-dev libboost-all-dev libhdf5-serial-dev libleveldb-dev liblmdb-dev libhdf5-dev pip install protobuf 复制代码 opencv这里把opencv单独拿出来说，是因为opencv的安装方法非常多 apt install python3-opencv conda install opencv 源码编译通过apt install安装最简单，也是最不容易出错的方法；其次是conda install，最容易出问题的是自己编译源码，编译参数复杂，依赖库繁多，而且还有版本差异。 安装完成后，建议使用opencv_version命令来查看当前版本，默认ubuntu 18.04源提供的是3.2.0版本，conda的会更高一些，这里是3.4.0，源码安装的话，注意在sudo make install后再执行一句sudo ldconfig。本文以conda的方式进行安装。 编译caffe接下来就可以编译caffe了 git clone https://github.com/BVLC/caffe.git cd caffe cp Makefile.config.example Makefile.config 复制代码编辑文件Makefile.config，主要是一些路径的修改，贴上已经修改好的 ## Refer to http://caffe.berkeleyvision.org/installation.html # Contributions simplifying and improving our build system are welcome! # cuDNN acceleration switch (uncomment to build with cuDNN). # 启用cuDNN加速 USE_CUDNN := 1 # CPU-only switch (uncomment to build without GPU support). # CPU_ONLY := 1 # uncomment to disable IO dependencies and corresponding data layers # 启用opencv USE_OPENCV := 1 # USE_LEVELDB := 0 # USE_LMDB := 0 # This code is taken from https://github.com/sh1r0/caffe-android-lib USE_HDF5 := 1 # uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary) # You should not set this flag if you will be reading LMDBs with any # possibility of simultaneous read and write # ALLOW_LMDB_NOLOCK := 1 # Uncomment if you're using OpenCV 3 # opencv大版本号是3，这里一定要注意 OPENCV_VERSION := 3 # To customize your choice of compiler, uncomment and set the following. # N.B. the default for Linux is g++ and the default for OSX is clang++ # CUSTOM_CXX := g++ # CUDA directory contains bin/ and lib/ directories that we need. CUDA_DIR := /usr/local/cuda # On Ubuntu 14.04, if cuda tools are installed via # "sudo apt-get install nvidia-cuda-toolkit" then use this instead: # CUDA_DIR := /usr # CUDA architecture setting: going with all of them. # For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility. # For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility. # For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility. # -gencode arch=compute_20,code=sm_21 # CUDA_ARCH := -gencode arch=compute_20,code=sm_20 # 这里使用的是CUDA10.1，所以要注释掉前两行 CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \ -gencode arch=compute_35,code=sm_35 \ -gencode arch=compute_50,code=sm_50 \ -gencode arch=compute_52,code=sm_52 \ -gencode arch=compute_60,code=sm_60 \ -gencode arch=compute_61,code=sm_61 \ -gencode arch=compute_61,code=compute_61 # BLAS choice: # atlas for ATLAS (default) # mkl for MKL # open for OpenBlas BLAS := atlas # Custom (MKL/ATLAS/OpenBLAS) include and lib directories. # Leave commented to accept the defaults for your choice of BLAS # (which should work)! # BLAS_INCLUDE := /path/to/your/blas # BLAS_LIB := /path/to/your/blas # Homebrew puts openblas in a directory that is not on the standard search path # BLAS_INCLUDE := $(shell brew --prefix openblas)/include # BLAS_LIB := $(shell brew --prefix openblas)/lib # This is required only if you will compile the matlab interface. # MATLAB directory should contain the mex binary in /bin. # MATLAB_DIR := /usr/local # MATLAB_DIR := /Applications/MATLAB_R2012b.app # NOTE: this is required only if you will compile the python interface. # We need to be able to find Python.h and numpy/arrayobject.h. # Python头文件路径，再加上numpy的头文件路径 PYTHON_INCLUDE := /home/xugaoxiang/anaconda3/include/python3.7m \ /home/xugaoxiang/anaconda3/lib/python3.7/site-packages/numpy/core/include # Anaconda Python distribution is quite popular. Include path: # Verify anaconda location, sometimes it's in root. # ANACONDA_HOME := $(HOME)/anaconda # PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ # $(ANACONDA_HOME)/include/python2.7 \ # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include # Uncomment to use Python 3 (default is Python 2) # 默认是python2，这里使用python3，一定要改，不然后面会报错相应没人在用python2了吧 PYTHON_LIBRARIES := boost_python3 python3.7m # PYTHON_INCLUDE := /usr/include/python3.5m \ # /usr/lib/python3.5/dist-packages/numpy/core/include # We need to be able to find libpythonX.X.so or .dylib. # libpython*.so库的路径 PYTHON_LIB := /home/xugaoxiang/anaconda3/lib # 如果设置了ANACONDA_HOME环境变量，可以使用下面的设置方法，作用一样 # PYTHON_LIB := $(ANACONDA_HOME)/lib # Homebrew installs numpy in a non standard path (keg only) # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include # PYTHON_LIB += $(shell brew --prefix numpy)/lib # Uncomment to support layers written in Python (will link against Python libs) # WITH_PYTHON_LAYER := 1 # Whatever else you find you need goes here. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /home/xugaoxiang/anaconda3/include LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/ # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies # INCLUDE_DIRS += $(shell brew --prefix)/include # LIBRARY_DIRS += $(shell brew --prefix)/lib # NCCL acceleration switch (uncomment to build with NCCL) # https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0) # USE_NCCL := 1 # Uncomment to use `pkg-config` to specify OpenCV library paths. # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.) # 启用pkg_config，方便caffe找到opencv USE_PKG_CONFIG := 1 # N.B. both build and distribute dirs are cleared on `make clean` # 默认编译的目录，所有的目标文件、可执行文件、库都存放在这里 BUILD_DIR := build DISTRIBUTE_DIR := distribute # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171 # 是否打开debug信息 # DEBUG := 1 # The ID of the GPU that 'make runtest' will use to run unit tests. TEST_GPUID := 0 # enable pretty build (comment to see full commands) Q ?= @ 复制代码完成后，执行 make all -j12 复制代码参数-j指的是使用多少个cpu核心，目的是加快编译速度，根据自己的实际情况设定 为了能在python中调用caffe，还需要执行 make pycaffe -j12 复制代码至此，整个编译就结束了。 验证使用ipython环境测试
如果细心一点，会发现，新开一个terminal，同样打开ipython，同样import caffe，但是会报错，这是什么原因？
在不报错的terminal中，查看环境变量就会发现端倪
在编译caffe的过程中，会export环境变量PYTHONPATH，所以我们在使用前也需要这样做
为了简便，可以将声明语句写入~/.bashrc中，就不用每次都去执行了 export PYTHONPATH=/home/xugaoxiang/Works/github/caffe/python:$PYTHONPATH 复制代码 Q & A Q1编译过程碰到了tiff相关的错误
这是由于之前opencv源码编译引起的，这里特别要注意一点，如果是从源码开始编译opencv，那么在配置的时候一定要加上选项-D BUILD_TIFF=ON。还有就是尽量不要同时拥有apt和conda安装的2种环境，对新手来说比较容易出错。 Q2进入ipython中，import caffe报错
将libhdf5_hl.so.100的路径加入LD_LIBRARY_PATH中 export LD_LIBRARY_PATH=/home/xugaoxiang/anaconda3/lib:$LD_LIBRARY_PATH 复制代码 Q3进入ipython中，import caffe报错
修改Makefile.config，修改PYTHON_LIBRARIES为 PYTHON_LIBRARIES := boost_python3 python3.7m 复制代码默认的是python2 Q4关于caffe中使用源码编译的opencv4，由于opencv4的版本差异，都会报错
这是由于在opencv4中，原来版本中的宏CV_LOAD_IMAGE_COLOR和CV_LOAD_IMAGE_GRAYSCALE已经改成了cv::IMREAD_COLOR和cv::ImreadModes::IMREAD_GRAYSCALE，所以，需要在caffe源码目录中查找并替换，才能够编译成功
编译过程中报hdf5头文件找不到？即使是执行了安装命令也一样 sudo apt install libhdf5-dev 复制代码 src/caffe/layers/hdf5_data_layer.cu:10:10: fatal error: hdf5.h: No such file or directory #include "hdf5.h" ^~~~~~~~ compilation terminated. Makefile:604: recipe for target '.build_release/cuda/src/caffe/layers/hdf5_data_layer.o' failed make: *** [.build_release/cuda/src/caffe/layers/hdf5_data_layer.o] Error 1 make: *** Waiting for unfinished jobs.... 复制代码这时候需要修改Makefile.config文件，在INCLUDE_DIR后添加路径/usr/include/hdf5/serial，在LIBRARY_DIR后添加/usr/lib/x86_64-linux-gnu/hdf5/serial 参考资料 anaconda基本使用 ubuntu安装CUDA和cuDNN opencv源码编译，支持CUDA github.com/BVLC/caffe/… |
【本文地址】
今日新闻 |
推荐新闻 |